HW 08 - Comparing proportions

Individual assignment

As usual, you have the following resources available to you:

• Please submit your lab using this link.
• Would you like to use one of your tokens for an extension or an upgrade from R (revise) to an M (meeting specs)? Please fill in this form.
• If you have questions, please book a slot during Ofer’s office hours!

Sex discrimination case study

Women get paid less than men. Can we use statistics to show that this is due to discrimination and not due to other factors?

We consider a study investigating sex discrimination in the 1970s, which is set in the context of personnel decisions within a bank. The research question we hope to answer is, “Are individuals who identify as female discriminated against in promotion decisions made by their managers who identify as male?”

The sex_discrimination data set can be found in the openintro R package. The data is based on Rosen B and Jerdee T. (1974). “Influence of sex role stereotypes on personnel decisions”. Journal of Applied Psychology 59(1):9-14.

To load the data, please install the package openintro, and then assign the dataset to the variable promo like so:

promo <- openintro::sex_discrimination

This study considered sex roles, and only allowed for options of “male” and “female”. Note that the identities being considered are not gender identities and that the study allowed only for a binary classification of sex.

Observed data

The participants in this study were 48 bank supervisors who identified as male, attending a management institute at the University of North Carolina in 1972. They were asked to assume the role of the personnel director of a bank and were given a personnel file to judge whether the person should be promoted to a branch manager position. The files given to the participants were identical, except that half of them indicated the candidate identified as male and the other half indicated the candidate identified as female.

These files were randomly assigned to the subjects. For each supervisor both the sex associated with the assigned file and the promotion decision were recorded. Using the results of the study, we would like to evaluate if individuals who identify as female are unfairly discriminated against in promotion decisions.

In this study, a smaller proportion of female identifying applications were promoted than males, but it is unclear whether the difference provides convincing evidence that individuals who identify as female are unfairly discriminated against.

The sex discrimination study can be thought of as 48 red and white cards.

The data are visualized as a set of cards. Note that each card denotes a personnel file (an observation from our dataset) and the colors indicate the decision: red for promoted and white for not promoted. Additionally, the observations are broken up into groups of male and female identifying groups.

• \(H_0:\) Null hypothesis. The variables sex and decision are independent. There is no real association between the two, and the observed difference between the proportion of males and females promoted, is just a coincidence. If we ran the experiment again with another 48 male bank supervisors, the results could be completely different.
• \(H_A:\) Alternative hypothesis. The variables sex and decision are not independent. The difference in promotion rates was not due to pure coincidence, and equally qualified female personnel are less likely to be promoted than male personnel.

Hypothesis testing.

These hypotheses are part of what is called a hypothesis test. A hypothesis test is a statistical technique used to evaluate competing claims using data. Often times, the null hypothesis takes a stance of no difference or no effect. This hypothesis assumes that any differences seen are due to the variability inherent in the population and could have occurred by random chance.

If the null hypothesis and the data notably disagree, then we will reject the null hypothesis in favor of the alternative hypothesis.

We will choose between the two competing claims by assessing if the data conflict so much with \(H_0\) that the null hypothesis cannot be deemed reasonable. If data and the null claim seem to be at odds with one another, and the data seem to support \(H_A,\) then we will reject the notion of independence and conclude that the data provide evidence of discrimination.

1. Is this an observational study or an experiment? How does the type of study impact what can be inferred from the results? Please justify your answer.

2. Create a contingency table, showing the percent of male and female candidates and promoted and not promoted. What is the difference in the proportion of males promoted and the proportion of females promoted? To find out, you would need to install the package janitor and to load it using the function library(janitor). Then, run the following commands:

# use the name of the two variables and put them 
# within the tabyl(____, ____) function. 
# Make sure you have one row for males and one for females.
promo %>% tabyl(____, ____) 

# Do the same thing, but this time show the percentages. 
# Make sure that the table shows the percentage 
# of males and females promoted. The percentage of 
# promoted and non-promoted should add up to 100%
# for males and for females. 
promo %>% tabyl(____, ____) %>% 
  adorn_percentages() %>% 
  adorn_pct_formatting()  

Variability of the statistic

The sex discrimination data is shuffled and reallocated to new groups of male and female files.

The contingency table shows the proportion of bank supervisors who recommended promotion and those who did not. Now, suppose the bankers’ decisions were independent of the sex of the candidate. Then, if we conducted the experiment again with a different random assignment of sex to the files, differences in promotion rates would be based only on random fluctuation in promotion decisions. We can actually perform this randomization, which simulates what would have happened if the bankers’ decisions had been independent of sex but we had distributed the file sexes differently.

The test procedure we employ in this section is sometimes referred to as a permutation test. In the simulation, we thoroughly shuffle the 48 personnel files, 35 labelled promoted and 13 labelled not promoted, together and we deal files into two new stacks. Note that by keeping 35 promoted and 13 not promoted, we are assuming that 35 of the bank managers would have promoted the individual whose content is contained in the file independent of the sex indicated on their file. We will deal 24 files into the first stack, which will represent the 24 “female” files. The second stack will also have 24 files, and it will represent the 24 “male” files. The figure highlights both the shuffle and the reallocation to the sham sex groups.

Then, as we did with the original data, we tabulate the results and determine the fraction of personnel files designated as “male” and “female” who were promoted.

Since the randomization of files in this simulation is independent of the promotion decisions, any difference in promotion rates is due to chance.

3. Use the code below to generate one permutation of the data. Then create a contingency table of that permutation, and compare your results to the previous contingency table you made above.

   • Are the total number of males and females the same as before? What about the total number of promoted and non-promoted?
   • How do the proportions of promotions compare?
   • How does the difference in the proportion of promotions compare?

# Create one permuatation - start by setting a seed.
set.seed(___)
# Assume that the decision to promote is independent of 
# the sexual identity of the candidate 
permute1 <- promo %>%
  specify(decision ~ sex, success = "promoted") %>%
  hypothesize(null = "independence") %>%
  generate(reps = 1, type = "permute")  

# Show the permutations generated
permute1 %>% tabyl(___, ___)

# And now with percentages...
permute1 %>% tabyl(____, ____) %>% 
  adorn____() %>% 
  adorn____()  

4. Finally, run the code below to simulate 1000 permutations of the data, permutations we would expect assuming that the null hypothesis was true.

   • Assuming independence, what is the 95% confidence interval of the distribution of differences in proportion?
   • Assuming independence, is the p-value of observing a difference in proportions as extreme as the one observed in our data?
   • Does our data support our hypothesis \(H_0\)? Why or why not?

# Create 1000 permutations, assuming that
# Start by setting a seed.
set.seed(___)
# Assume that the decision to promote is independent of 
# the sexual identity of the candidate 
permute1000 <-  promo %>%
  # in the formula we use `dependent` ~ `independent`
  specify(decision ~ sex, success = "promoted") %>%
  hypothesize(null = "independence") %>%
  generate(reps = ___, type = "permute") %>%
  calculate(stat = "diff in props", order = c("male", "female")) 

# Create a histogram of the proportions, assuming 
ggplot(___$___) + geom_____()

# Now find the 95% confidence interval 
quantile(___$___, probs=c(___,___))

# Calculate the p-value, i.e. the probability that, given 
# the independence assumption (promotion vs sex) we 
# will observe a difference in proportion as extreme 
# as the one observed in our dataset
p.value <- mean(___$___ > ___)

Opportunity cost case study

How rational and consistent is the behavior of the typical American college student? In this section, we’ll explore whether college student consumers always consider the following: money not spent now can be spent later.

In particular, we are interested in whether reminding students about this well-known fact about money causes them to be a little thriftier. A skeptic might think that such a reminder would have no impact. We can summarize the two different perspectives using the null and alternative hypothesis framework.

• \(H_0:\) Null hypothesis. Reminding students that they can save money for later purchases will not have any impact on students’ spending decisions.
• \(H_A:\) Alternative hypothesis. Reminding students that they can save money for later purchases will reduce the chance they will continue with a purchase.

In this section, we’ll explore an experiment conducted by researchers that investigates this very question for students at a university in the southwestern United States. The study was published under Frederick S, Novemsky N, Wang J, Dhar R, Nowlis S. 2009. Opportunity Cost Neglect. Journal of Consumer Research 36: 553-561.

Observed data

One-hundred and fifty students were recruited for the study, and each was given the following statement:

Imagine that you have been saving some extra money on the side to make some purchases, and on your most recent visit to the video store you come across a special sale on a new video. This video is one with your favourite actor or actress, and your favorite type of movie (such as a comedy, drama, thriller, etc.). This particular video that you are considering is one you have been thinking about buying for a long time. It is available for a special sale price of $14.99. What would you do in this situation? Please circle one of the options below.

Half of the 150 students were randomized into a control group and were given the following two options:

1. Buy this entertaining video.

2. Not buy this entertaining video.

The remaining 75 students were placed in the treatment group, and they saw a slightly modified option (B):

1. Buy this entertaining video.

2. Not buy this entertaining video. Keep the $14.99 for other purchases.

Would the extra statement reminding students of an obvious fact impact the purchasing decision? To answer the question, we will define a success in this study as a student who chooses not to buy the video (Success is often defined in a study as the outcome of interest, and a “success” may or may not actually be a positive outcome. For example, researchers working on a study on COVID prevalence might define a “success” in the statistical sense as a patient who has COVID-19.)

Then, the value of interest is the change in video purchase rates that results by reminding students that not spending money now means they can spend the money later.

The opportunity_cost data can be found in the openintro R package.

5. Is this an observational study or an experiment? How does the type of study impact what can be inferred from the results? Please justify your answer.

6. Create a contingency table, showing the treatment and control group choosing and the succes rate for each group. What is the difference in the success rate?

7. Generate one permutation of the data, just as we did above. Then create a contingency table of that permutation, and compare your results to the previous contingency table you made above.

   • Are the total number of participants in each group the same as before? What about the total number of successes?
   • How do the rate of success in your permutation compare to that of the study?
   • How does the difference in the rate of success in your permutation compare to that of the study?

8. Finally, run the code below to simulate 1000 permutations of the data, permutations we would expect assuming that the null hypothesis were true.

   • Assuming independence, what is the 95% confidence interval of the distribution of differences in the rate of success?
   • Assuming independence, is the p-value of observing differences in the rate of success as extreme as the one observed in our data?
   • Does our data support our hypothesis \(H_0\)? Why or why not?

Numeric literacy: A rising tide lifts all boats?

Poor students in rich countries and rich students in poor ones. Who has better math scores?

There are some things money can’t buy. Education, however, does not appear to be among them—at least as measured by performance on international exams. On average, the poorest pupils in wealthy countries obtain vastly higher test scores than the richest in developing ones. In turn, strong students tend to become productive workers, making the mostly rich economies richer still.

The exact mechanism by which knowledge is bought remains unclear. Do students in the rich world fare better because their governments provide superior schools? Or is the reason that they tend to have richer parents, and enjoy more educational resources at home? A new working paper by Dev Patel of Harvard University and Justin Sandefur of the Centre for Global Development, a think-tank, offers strong evidence that the wealth of a country affects exam results just as much as the wealth of a pupil’s household does.

Evaluating test scores around the world is harder than it sounds. Although pupils in the rich world mostly take one of a few big international exams, many developing countries rely on regional tests. This precludes apples-to-apples comparisons.

Please study the graph below carefully, read the explanation and answer the questions below:

Sources: The Economist, Sources: “A Rosetta Stone for Human Capital”, by D. Patel and J. Sandefur; World Bank; The Economist

This article appeared in the Graphic detail section section of the print edition under the headline “A rising tide”.

Questions:

1. What do you notice? In your answer, please refer to all three graphs.

   After studying ALL THREE graphs and reading the text, what details do you see in the graph that are unusual or surprising? Looking at it carefully and paying attention to detail, what can you get out of the graph that is not necessarily obvious at first glance.

2. What do you wonder?

   Try to formulate a question, related to the graph, something that you would genuinely like to know. Please state your question clearly: as in a statement that ends with a question mark.

3. How does this relate to you and your community?

   Think of something that makes this graph relate to you personally or to your community, however defined (nationality, age, origin, gender, etc..). Make sure that you do not describe something that relates to everyone, no matter who they are, but only to members of your community. You are looking for something specific that does not relate to other communities. This is your opportunity to be creative!

4. What’s going on in this graph? Write a catchy headline that captures the graph’s main idea.